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DETAILED ACTION 



Specification 

1 . Applicant is reminded of the proper language and format for an abstract of the 
disclosure. 

The abstract should be in narrative form and generally limited to a single paragraph on a separate 
sheet within the range of 50 to 150 words. It is important that the abstract not exceed 1 50 words in length 
since the space provided for the abstract on the computer tape used by the printer is limited. The form 
and legal phraseology often used in patent claims, such as "means" and "said," should be avoided. The 
abstract should describe the disclosure sufficiently to assist readers in deciding whether there is a need 
for consulting the full patent text for details. 

The language should be clear and concise and should not repeat information given in the title. It 
should avoid using phrases which can be implied, such as, "The disclosure concerns," "The disclosure 
defined by this invention," "The disclosure describes," etc. 



2. The abstract of the disclosure is objected to because it is not narrative. It 
consists and has been drafted as one long run-on sentence, much like claim 1 , which is 
improper. The intent of the abstract is to give a concise but brief statement of the 
disclosure or the invention as a whole consisting of a series of complete sentences 
forming a single paragraph. 



Correction is required. See MPEP § 608.01(b). 



3. The disclosure is objected to because of the following informalities: 
Page 8, line 26: "FIG 3" should be ~ FIG 3b 



Appropriate correction is required. 
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Claim Objections 

4. Claims 4-9, and 1 1 are objected to because of the following informalities: 
Claims 4 and 5, line 3 respectively: The end of the claim should be followed by a 
period, "consecutive images" should be - consecutive images. 

Claims 6 and 7, line 3 respectively: "on the previous image" should be - on a 
previous image'-. 

Claims 8 and 9, lines 2-3 respectively: "on both the previous and" should be - on 
both a previous and -. 

Claim 1 1 , line 2: "using the belief propagation algorithm" should be - using a 
belief propagation algorithm -. 

Appropriate correction is required. 

Claim Rejections - 35 USC §112 

5. The following is a quotation of the second paragraph of 35 U.S.C. 112: 

The specification shall conclude with one or more claims particularly pointing out and distinctly 
claiming the subject matter which the applicant regards as his invention. 

6. Claims 7 and 9 are rejected under 35 U.S.C. 112, second paragraph, as being 
indefinite for failing to particularly point out and distinctly claim the subject matter which 
applicant regards as the invention. 

Re Claim 7: Claim 7 is the exact replicate of its dependent claim 6, making this 
claim indefinite. 
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Re Claim 9: Claim 9 is the exact replicate of its dependent claim 8, making this 
claim indefinite. 

Appropriate correction is required. 

Claim Rejections - 35 USC § 102 

7. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 1 02 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
States. 

8. Claims 1 , 2, and 5 are rejected under 35 U.S.C. 102(b) as being anticipated by 
Simpson ("A recurrent neural network classifier for improved retrievals of areal extent of 
snow cover" - IEEE - vol 39, Oct 2001 , pages 2135-2147). 

Re Claim 1: Simpson discloses a method / single image classification using feed- 
forward neural networks (FFNN) and image sequence classification using recurrent 
neural networks (RNNCCS) (see pages 2138-2139, section B. Present Approaches) for 
improving scene classification of a sequence of digital images / sequence of snow cover 
images comprising the steps of (a) providing a sequence of images captured in 
temporal succession / temporal sampling into time series (see page 2139, section 2 - 
Image Sequence Classification Using Recurrent Neural Networks, first paragraph); (b) 
classifying each of the images individually / Single Image Classification using Feed- 
Forward Neural Networks (FFNN) based on information / spectral and textural 
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contained in the individual image to generate a first image classification (see page 
2138, section 1 - Single Image Classification Using Feed-Forward Neural Networks, first 
paragraph); and (c) imposing a pre-determined temporal context model / Recurrent 
Neural Network (RNNCCS) on the sequence of images / sequence of snow cover 
temporal sampling into time series images to generate a final image classification (see 
page 2139, section 2 - Image Sequence Classification Using Recurrent NeUral 
Networks, first paragraph, the RNNCCS uses "short term memory" of both texture and 
classification data from the previous image(s) to do a final classification on the current 
image being analyzed). 

Re Claim 2: Simpson discloses the information used in step (b) includes pixel 
information / spectral and textural (see page 2138, section 1 - Single Image 
Classification Using Feed-Forward Neural Networks, first paragraph). 

Re Claim 5: Simpson discloses the pre-determined temporal context model in step (c) is 
dependent on elapsed time / temporal sampling into time series between consecutive 
images (see page 2139, section 2 - Image Sequence Classification Using Recurrent 
Neural Networks, first paragraph, the RNNCCS uses the twelve daylight scenes per day 
sequence of images in a time series manner for further improvement in classification). 
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Claim Rejections - 35 USC § 103 

9. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

10. Claim 3 is rejected under 35 U.S.C. 103(a) as being unpatentable over Simpson 
in view of Tretter et al (US 6,977,679 B2). The teachings of Simpson are discussed 
above. 

However, Simpson fails to teach the information used in step (b) includes 
metadata information. 

Tretter, as recited in claim 3 . discloses the information used in step (b) includes 
metadata / focusing distance information (see abstract, lines 6-13). 

Therefore, in view of Tretter, it would have been obvious to one of ordinary skill in 
the art at the time the invention was made to modify Simpson's step (b) of classifying 
each image individually by including focusing distance metadata in order to enhance the 
classification of snow cover by further distinguishing clouds and snow cover by 
identifying the difference in distance between clouds and the snow cover from the 
satellite which captures images. 

1 1 . Claim 4 is rejected under 35 U.S.C. 103(a) as being unpatentable over Simpson. 
The teachings of Simpson are discussed above. 
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Although the pre-determined context model of Simpson is disclosed as 
depending on elapsed time between consecutive images and is not specifically 
disclosed, as recited in claim 4 , as being independent of elapsed time between 
consecutive images, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to have such a feature of independency in elapsed time 
because the elapsed time between Simpson's satellite consecutive images could be 
irregularly spaced or not evenly time spaced as long as the consecutive images are 
produced as twelve daylight scenes per day in order to capture the more detailed and 
more visible images of snow cover during one point of the day and fewer during the rest 
of the day. 

12. Claims 6-1 1 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Simpson in view of Huang ("Integration of multimodal features for video scene 
classification based on HMM" - IEEE - Sept 1999, pages 53-58). The teachings of 
Simpson are discussed above. 

However, Simpson fails to teach that the temporal context model is a causal 
Hidden Markov Model, that the temporal context model is a non-causal model, that the 
temporal context model is imposed using a Viterbi algorithm, and that the temporal 
context model is imposed using a belief propagation algorithm. 

Huang, as recited in claims 8 and 9 respectively , discloses the pre-determined 
temporal context model is a non-casual model / discrete ergodic Hidden Markov Model 
dependent on both the previous and subsequent images / visited from any state (see 
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page 55, section Product HMM, page 56, section SIMULATION RESULTS, second 
paragraph, the discrete ergodic HMM visits any states or images from any state or 
image which makes the model non-causal). 

Therefore, in view of Huang, it would have been obvious to one of ordinary skill in 
the art at the time the invention was made to modify Simpsons temporal context model 
by replacing it with a non-causal discrete ergodic Hidden Markov Model in order to give 
more correlation data between all the images of the sequence instead of just a few 
images (images before the current image) to further improve the accuracy of the 
classification. 

Although Simpson's temporal context model in view of Huang's modifications 
teaches a non-causal discrete ergodic Hidden Markov Model, it does not specifically 
disclose, as recited in claims 6 and 7 respectively , that the temporal context model is 
the causal Hidden Markov Model dependent on the previous image, and it does not 
specifically disclose, as recited in claim 10 , that the temporal context model is imposed 
using Viterbi algorithm, and it does not specifically disclose, as recited in claim 1 1 , that 
the temporal context model is imposed using a belief propagation algorithm. It would 
have been obvious to one of ordinary skill in the art at the time the invention was made 
though to have such a feature of causality in a HMM model, a Viterbi algorithm, or a 
belief propagation algorithm for a temporal context model because they are just other 
methods of computing the probability for classification of a particular sequence which 
Huang's non-causal Hidden Markov Model is basically accomplishing. 
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Conclusion 

13. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. Kowald et al discloses a visual language classification system; 
Foote et al discloses methods and apparatuses for video segmentation, classification, 
and retrieval using image class statistical models; Foote et al discloses methods and 
apparatuses for interactive similarity searching, retrieval, and browsing of video; 
Nicponski et al discloses a method and system for processing images for themed 
imaging services; Nefian et al discloses a coupled hidden markov model for audiovisual 
speech recognition; Niikura et al discloses a method and device for classifying video 
blocks; Huang discloses joint video scene segmentation and classification based on 
hidden Markov model; Szummer discloses indoor-outdoor image classification; 
Schuster discloses fast online video image sequence recognition with statistical 
methods; Morguet discloses feature extraction methods for consistent spatio-temporal 
image sequence classification using hidden Markov models; Saitwal discloses a 
multichannel temporally adaptive system for continuous cloud classification from 
satellite imagery; Savakis discloses indoor vs. outdoor classification of consumer 
photographs using low-level and semantic features; Vasconcelos discloses a Bayesian 
framework for semantic content characterization; Baik discloses online model 
modification for adaptive texture recognition in image sequences; Kane discloses 
Bayesian network structure learning and inference in indoor vs. outdoor image 
classification. 
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14. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Bernard Krasnic whose telephone number is (571 ) 270- 
1357. The examiner can normally be reached on Mon-Thur 8:00am-3:00pm and every 
other Friday 8:00am-3:00pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Jong-Suk (James) Lee can be reached on (571) 272-7044. The fax phone 
number for the organization where this application or proceeding is assigned is 571- 
273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

Bernard Krasnic 
January 17, 2007 



JONG SUK LEE 
SUPERVISORY PATENT EXAMINER 




